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Abstract 

The  intent  of  this  research  is  to  introduce  forward-looking  metrics  and  take  the 
first  steps  in  developing  a  forward-looking  metric  for  Air  Force  sustainment  operations. 
Specifically,  this  research  seeks  to  answer  four  research  questions  addressing  what  the 
Air  Force  is  currently  using  for  sustainment  metrics,  what  are  forwarding  looking 
metrics,  how  can  a  forward-looking  metrics  be  developed  to  mitigate  current  sustainment 
operations  shortfalls,  and  how  can  they  be  used  to  optimize  stock  levels,  within  a  given 
set  of  constraints,  based  upon  a  changing  MICAP-hour  requirement.  These  questions  are 
answered  through  a  comprehensive  literature  review,  interviews  with  subject  matter 
experts  and  the  development  of  an  integer  linear  program.  The  research  identified  a  need 
to  sub-divide  the  MICAP  hours  metric  into  three  categories,  using  a  robust  data  selection 
process:  recurring,  non-recurring  and  in-inventory  MICAP  hours.  From  this,  four 
primary  models  are  developed,  which  culminated  in  an  integer  linear  program  to 
determine  the  optimal  stock  level  increases  to  improve  (decrease)  monthly  recurring 
MICAP  hours,  subject  to  the  lowest  cost  with  the  highest  effectiveness.  The  C-5  Galaxy 
aircraft  fleet  sustainment  data  is  used  to  develop  this  forward-looking  metric. 

This  research  proves  forward-looking  metrics  are  a  viable  tool  for  the  future  of 
sustainment  operations  and  should  be  leveraged  in  combination  with  technology  to  be  the 
most  effective  and  efficient  Air  Force  in  the  world. 
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ADVANCING  FORWARD-LOOKING  METRICS:  A  LINEAR  PROGRAM 
OPTIMIZATION  AND  ROBUST  VARIABLE  SELECTION  FOR  CHANGE  IN 

STOCK  LEVELS  AS  A  RESULT  OF  RECURRING  MICAP  PARTS 

I.  Introduction 

General  Issue  and  Problem  Statement 

The  United  States  is  proud  to  own  the  largest  air  force  in  the  world,  with  5,484 
aircraft,  450  ICBMs  and  63  satellites  (Almanac,  2012),  but  sustaining  this  fleet  has  many 
challenges.  Specifically,  the  Air  Force  supply  system  is  fraught  with  outdated 
technology,  complex  processes  and  a  myriad  of  metrics  that  are  difficult  to  interpret. 

Current  Air  Force  metrics  used  to  assess  aircraft  fleet  sustainment  health  are 
complex,  do  not  provide  actionable  decision  options  and  are  primarily  “historical” 
metrics.  Air  Force  metrics  primarily  show  what  happened  after  the  fact  for  post¬ 
operation  trend  analysis,  including:  issue  effectiveness,  stockage  effectiveness, 
cannibalization  rates,  average  repair  cycle  days,  etc.  Conversely,  forward-looking 
metrics  are  measures  that  identify  deviations  from  a  desired  state  before  it  occurs  or 
enable  a  desired  state  to  be  reached  through  an  understanding  of  what  controllable  system 
inputs  map  into  the  metric  function.  The  Air  Force  is  not  currently  using  forward- 
looking  metrics,  but  has  taken  initial  steps  in  developing  “scorecards”  and  “dashboards,” 
which  enable  a  smooth  transition  into  forward-looking  metrics.  Forward-looking  metrics 
could  greatly  support  the  Air  Force  by  providing  a  real-time  assessment  and  identify 
areas  of  improvement;  they  will  help  manage  problems  before  they  become  serious. 
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As  the  Air  Force’s  budget  continues  to  shrink,  Air  Force  leadership  needs  to 
pinpoint  their  money  and  efforts  on  areas  to  improve  aircraft  fleet  health  the  most.  A 
forward  looking  metric  is  needed  to  enable  this  capability  for  aircraft  sustainment. 

Research  Questions 

The  objective  of  this  research  is  to  begin  to  answer  the  question  posed  by  the  Air 
Force  Sustainment  Center,  “Can  forward-looking  metrics  be  used  in  Air  Force 
sustainment  operations  and  if  so,  how?”  Since  the  question  is  extremely  broad,  this 
research  will  focus  on  developing  a  forward-looking  metric  for  Air  Force  stock  levels,  as 
related  to  a  changing  Mission  Capability  (MICAP)  hours.  The  intent  of  this  research  is  to 
introduce  forward-looking  metrics  into  Air  Force  sustainment  operations  and  take  the 
first  steps  necessary  in  developing  a  forward-looking  metric.  The  following  investigative 
questions  will  be  analyzed: 

1.  What  metrics  are  currently  used  for  sustainment  operations  in  the  Air  Force? 

2.  What  are  forward-looking  metrics? 

3.  Can  forward-looking  metrics  be  developed  to  mitigate  current  sustainment 
operation  shortfalls? 

4.  How  can  forward-looking  metrics  be  used  to  optimize  stock  levels,  within  a  given 
set  of  constraints,  based  upon  a  changing  MICAP-hour  requirement? 

Methodology  Overview 

This  research  leveraged  multiple  research  practices  to  answer  these  research 
questions,  including  current  and  historical  literature  review,  interviews  with  sustainment 
operations  experts  and  linear  programming.  Ultimately,  analysis  of  resource  allocation 
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using  linear  programming  was  used  to  gain  insight  into  how  to  effectively  influence 
future  sustainment  decision  making  for  aircraft  parts.  The  C-5  Galaxy  fleet’s  supply  and 
maintainability  data  was  be  used  to  build  the  optimization  program  using  Excel  Solver. 

Assumptions/Limitations 

The  researcher  assumes  the  data  obtained  from  the  Air  Force  Supply  and 
Maintenance  systems  was  valid  and  accurate  data.  Data  was  obtained  from  the 
Requirements  Management  System  (D200). 

To  simplify  the  data  sets  into  readily  useable  data,  an  assumption  was  made  that 
all  MICAPs  were  satisfied  within  the  same  month  they  went  MICAP.  If  a  part  was 
MICAP  and  counted  in  two  or  more  sequential  months,  it  appears  as  separate  MICAPs. 

Limitations  to  this  research  arose  from  availability  of  data;  therefore  the 
methodology  was  shaped  around  what  data  was  available.  Additionally,  there  was 
limited  information  available  on  sustainment  metrics  used  in  the  civilian  aviation 
industry.  This  is  likely  due  to  proprietary  information  not  being  publically  available. 
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II.  Literature  Review 


Chapter  Overview 

This  chapter  provides  an  overview  of  recent  literature  and  research  on  forward- 
looking  metrics,  sustainment  metrics  and  Air  Force  sustainment  initiatives.  This  is  not 
intended  to  be  an  exhaustive  review,  but  to  provide  an  introduction  to  forward-looking 
metrics  and  an  overview  of  metrics  as  they  pertain  to  sustainment  operations  within  the 
Air  Force. 

Metrics 

Metrics  can  be  classified  into  one  of  three  categories:  Historical  Metrics,  Real- 
Time  (Scorecard  or  Dashboard)  Metrics  and  Forward-Looking  Metrics.  For  the  intent  of 
this  research,  the  following  definitions  apply  to  these  categories  of  metrics. 

Historical,  or  rear-looking  metrics,  are  measures  that  give  data  on  what  happened 
after  it  occurred  (Adler,  2003).  Examples  in  civilian  industry  include:  sales,  profits, 
return  on  investment,  number  of  customers,  costs,  etc.  Within  the  Air  Force,  most  of  the 
metrics  are  historical;  examples  include:  issue  and  stockage  effectiveness,  aircraft 
availability,  mission  capable  rate,  etc. 

Real-time  metrics  are  usually  referred  to  as  a  scorecard  or  dashboard  metrics.  The 
Scorecard  (known  as  a  Dashboard  within  the  Air  Force  community)  framework  for 
Performance  Measures  (PMs)  was  introduced  by  Kaplan  and  Norton  in  1996,  which  they 
called  the  “Balanced  Scorecard  (BSC).”  The  BSC  framework  is  built  around  four 
perspectives:  1)  financial,  2)  customers,  3)  internal  processes,  and  4)  learning  and  growth 
(Kaplan  &  Norton,  1996).  The  BSC  incorporates  traditional  financial  performance 
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measures  (historical  metrics)  with  non-financial  strategic  goals  to  give  a  more  “balanced” 
view  of  operational  performance  (BSC  Resources,  2013).  This  concept  has  evolved  to 
redefine  PMs  and  metrics  into  more  realistic  plans  to  achieve  strategic  targets 
(Gunasekaran  &  Kobu,  2007). 

Forward-looking  metrics  are  measures  that  identify  deviations  from  a  desired  state 
before  it  occurs  or  enable  a  desired  state  to  be  reached  through  an  understanding  of  what 
controllable  system  inputs  map  into  the  metric  function.  They  describe  what  will  occur 
before  it  happens.  Although  it  can  be  referred  to  as  a  predictive  measure,  the  intent  of  a 
forward-looking  metric  is  more  than  that;  it  is  one  that  incorporates  predictive  indicators 
into  a  useable  data  set  for  analysis.  For  example,  a  three  phased-approach  of  (1)  using 
trend  analysis  on  historical  data,  (2)  incorporating  a  known  future  requirement  and  then 
(3)  determining  the  gap  between  the  two  is  a  good  path  in  building  a  forward-looking 
metric  (Adler,  2003).  This  will  be  discussed  further  in  a  following  section. 

Forward-Looking  Metrics 

Development  of  forward-looking  metrics  is  a  long  standing  quest  within 
the  Department  and  remains  in  the  forefront  of  Congressional  interest. . . 

The  key  elements  of  a  [forward-looking  metric]  are  a  clean  input  signal,  a 
short  term  predictive  feedback  loop  and  a  long  term  feedback  loop  to 
continually  improve  the  predictive  metric.  (Sadauskas,  2009) 

Although  forward-looking  metrics  have  only  recently  come  onto  the  military’s 
horizon  of  research  within  the  past  few  years,  civilian  industry  has  been  researching 
forward-looking  metrics  extensively  in  business  marketing,  sales  forecasting  and 
customer  loyalty.  Since  neither  of  these  subject  areas  applies  directly  to  sustainment 
operations,  only  one  study  will  be  discussed,  which  highlights  the  majority  of  the 
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research  performed.  In  2006,  a  team  of  six  researchers  studied  forward  looking  metrics, 
which  they  termed  Adaptive  Foresight.  Although  their  research  was  on  customer  metrics, 
the  foundation  of  their  research  on  forward-looking  metrics  can  be  translated  for  use  with 
the  Air  Force’s  sustainment  of  its  aircraft. 

Zeithaml  et  al  (2006)  studied  customer  metrics  to  develop  adaptive  foresight  for 
anticipating  changes  in  the  marketplace  to  influence  (or  increase)  a  company’s  customer 
base.  The  researchers  found  the  metrics  being  used  were  primarily  historical  metrics,  and 
by  the  time  the  data  was  received  and  processed,  it  merely  represented  what  happened  in 
the  past. 

Some  companies  determined  these  metrics  were  insufficient  in  assessing  their 
customers  and  transitioned  to  Dashboard  metrics.  The  most  common  dashboard  for 
customer  metrics  is  called  the  Customer  Relationship  Management  (CRM)  metric.  Data 
for  the  CRM  metric  are  housed  in  large  databases  and  have  evolved  to  provide  a  near-real 
time  assessment.  But  even  these  fall  short  of  the  adaptive  foresight  (forward-looking 
metric)  being  sought  (Zeithaml,  Bolton,  Deighton,  Keiningham,  Lemon,  &  Petersen, 
2006). 

Zeithaml  et  al  (2006)  propose  forward-looking  metrics,  which  they  term 
“headlights,”  are  more  desirable  than  what  is  currently  available,  as  headlights  “project 
where  customers  are  going  rather  than  where  they  have  been.”  The  rear-looking 
(historical)  customer  metrics  focus  on  perceptions,  attitudes,  behavior  and  financial 
measures.  The  forward-looking  customer  metrics,  such  as  the  Customer  Lifetime  Value 
(CLV)  metric,  incorporates  additional  factors,  such  as  the  customer’s  characteristics, 
company’s  planned  marketing  strategy  and  environmental  factors. 
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The  study  continues  by  providing  a  four-step  approach  to  Adaptive  Foresight,  as 
shown  in  Figure  1. 


Figure  1:  Adaptive  Foresight:  The  Process  (Zeithaml,  Bolton,  Deighton,  Keiningham,  Lemon,  & 

Petersen,  2006) 

Step  1:  Developing  foresight  capability:  This  step  requires  the  company  be  able  to 
behold  the  possibility  of  different  futures  occurring. 

Step  2:  Generating  alternative  futures:  Develop  future  strategies/scenarios  for  both 

external  factors  (customer  or  market  changes)  and  internal  factors  (resources). 

Step  3:  Identifying  key  levers  to  influence  customers:  Identify  how  the  company  can 
influence  customers  within  those  given  scenarios  from  Step  2. 

Step  4:  Developing  offerings  that  fit  customers’  “futures”:  Developing  the  services  or 
products  identified  in  Step  3  that  will  influence  the  customer  within  the  given 
scenarios.  The  exact  future  is  unknown,  so  companies  should  determine  which 
potential  futures  identified  in  Step  2  are  more  likely  to  occur  and  focus  their 
energy  on  developing  the  key  levers  from  those  scenarios. 
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Sadauskas  (2009)  presents  a  more  concise  model,  as  shown  in  Figure  6,  in  an 
effort  to  “achieve  a  desired  effect  under  specified  standards  and  conditions  through  a 
combination  of  means  and  ways  across  the  [doctrine,  organization,  training,  materiel, 
leadership  and  education,  personnel  and  facilities]  DOTMLPF  to  perform  a  set  of  tasks  to 
execute  a  specific  course  of  action.” 


Figure  2:  Notional  model  for  Enhancing  Forward-Looking  Schedule  and  Performance  Predictors 

(Sadauskas,  2009) 


Using  Sadauskas  (2009)  research  on  forward-looking  capability  and  combining 
with  research  from  civilian  industry,  one  path  to  developing  forward-looking  metrics  can 
be  defined  as: 

Step  1 :  Determine  the  current  tasks  and  associated  metrics  currently  in  use  and 
how  they  influence  the  system  (Control  Signals). 
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Step  2:  Determine  the  forward-looking  capability  required  and  what  influences  it. 
Step  3:  Establish  a  framework  for  the  forward-looking  metric. 

Step  4:  Develop  a  feedback  loop  for  continual  adaptive  forecast. 


Sustainment  Metrics 

The  Weapon  Systems  Acquisition  Reform  Act  of  2009  includes  a 
Performance  Assessment  requirement  to  evaluate  the  extent  to  which 
current  metrics  are  likely  to  predict  a  timely  delivery  of  a  level  of 
capability  to  the  warfighter  that  is  consistent  with  the  level  of  resources  to 
be  expended  and  provides  superior  value  to  alternative  approaches  that 
may  be  available  to  meet  the  same  military  requirement.  (Sadauskas, 
2009) 


Civilian  Industry  Sustainment  Metrics 

There  are  limited  articles  on  supply  chain  management  PMs  and  metrics  within 
the  civilian  industry,  making  it  difficult  to  compare  the  civilian  aviation  industry 
sustainment  metrics  to  those  the  military  uses.  In  a  survey  of  literature  and  reported  case 
studies  between  1995  and  2004,  Gunasekaran  and  Kobu  (2007)  studied  the  importance  of 
PMs  and  metrics  to  identify  the  key  indicators  used  in  the  supply  chain  (SC)  and  logistics 
environments.  Twenty-six  metrics  were  identified,  as  listed  in  Table  1. 
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Table  1:  Metrics  used  to  measure  performance  in  SCM  systems  and  their  relations  to  categories  and 
factors  suggested  by  researchers  (Gunasekaran  &  Kobu,  2007) 


Tabic  3.  Metrics  used  to  measure  performance  in  SCM  systems  and  their  relations  to  categories  and  factors  suggested  by  researchers. 


Metrics 

A 

B 

C 

D 

E 

1 

( 

G 

Total 

Percentage 

1 

2 

3 

4 

1 

2 

3 

4 

1 

2 

3 

4 

5 

1 

2 

3 

1 

2 

1 

2 

1 

2 

01  Accuracy  of  scheduling 

X 

X 

X 

X 

X 

X 

X 

7 

32 

05  Bid  management  cycle  time 

X 

X 

X 

X 

X 

\ 

\ 

7 

32 

06  Capacity  utilization 

X 

X 

X 

X 

X 

X 

X 

X 

X 

9 

41 

07  Compliance  to  regulations 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

10 

45 

08  Conformance  to  specifications 

X 

X 

X 

X 

X 

X 

X 

X 

X 

9 

41 

18  Delivery  reliability 

X 

\ 

X 

X 

X 

\ 

X 

\ 

X 

9 

41 

24  Forecasting  accuracy 

X 

X 

X 

X 

X 

X 

X 

X 

X 

9 

41 

29  Inventory  costs 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

14 

63 

33  Labor  efficiency 

X 

X 

X 

\ 

X 

X 

X 

X 

8 

36 

35  Lead  lime  for  procurement 

X 

X 

X 

X 

X 

X 

X 

7 

32 

36  Lead  lime  manufacturing 

X 

X 

X 

X 

\ 

X 

X 

\ 

\ 

9 

41 

39  Obsolescence  cost 

X 

X 

X 

X 

X 

X 

X 

X 

8 

32 

44  Overhead  cost 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

10 

42 

46  Perceived  quality 

X 

X 

X 

X 

X 

X 

X 

7 

32 

47  Perceived  value  of  product 

\ 

X 

X 

X 

X 

X 

6 

27 

50  Process  cycle  time 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

11 

50 

51  Product  development  time 

X 

X 

X 

X 

X 

X 

X 

X 

8 

36 

54  Product/service  variety 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

10 

45 

55  Production  flexibility 

X 

X 

X 

X 

X 

\ 

X 

X 

X 

X 

X 

11 

50 

62  Return  on  investment 

X 

X 

X 

X 

X 

X 

X 

7 

32 

63  Selling  price 

X 

X 

X 

X 

X 

X 

X 

X 

X 

9 

41 

68  Stock  out  cost 

X 

X 

X 

X 

X 

X 

X 

X 

8 

32 

71  Supply  chain  response  time 

X 

X 

X 

X 

X 

X 

X 

X 

X 

\ 

X 

11 

50 

76  Transportation  cost 

X 

X 

X 

X 

\ 

X 

X 

X 

8 

32 

77  Value  added 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

10 

45 

81  Warranty  cost 

X 

X 

X 

X 

X 

X 

X 

7 

32 

Total 

10 

13 

7 

13 

12 

9 

12 

7 

13 

4 

9 

3 

7 

7 

9 

14 

9 

17 

22 

5 

11 

16 

Percentage 

38 

50 

27 

50 

46 

35 

46 

27 

50 

15 

35 

12 

27 

27 

35 

54 

35 

65 

85 

19 

42 

61 

A.  Balance  score  perspectives:  1.  Financial,  2.  Internal  process,  3.  Innovation  and  improvement.  4.  Customers.  B.  Components  of  performance  measures:  1.  Time, 
2.  Resource  utilization,  3.  Output.  4.  Flexibility.  C.  Location  of  measures  in  supply  chain  links:  I.  Planning  and  product  design.  2.  Supplier,  3.  Production,  4.  Delivery, 
5.  Customer.  D.  Decision  level:  1.  Strategic,  2.  Tactical,  3.  Operational.  E.  Financial  base:  1.  Financial,  2.  Non  financial.  F.  Measurement  base:  1.  Quantitative, 
2.  Nonquantitative.  G.  Traditional  vs.  Modern:  1.  Function  based.  2.  Value  hased.| 


Gunasekaran  and  Kobu  also  presented  six  observations  as  they  relate  to  these  key 
performance  indicators  (KPI),  as  follows: 


1.  Internal  business  process  (50%  of  the  KPI)  and  customers  (50%  of  the 
KPI)  play  a  significant  role  in  SC  environments.  This  implies  that  internal 
business  process  PMs  have  significant  impact  on  the  operational 
performance. 

2.  The  most  widely  used  PM  is  financial  performance  (38%  of  the  KPI). 

This  indicates  that  we  cannot  ignore  the  fact  that  still  cost  plays  a  major 
role  in  a  SC  environment.  However,  nonfinancial  performance  measures 
are  important  for  measuring  the  operational  performance. 

3.  Innovation  and  process  improvement  constitutes  27%  of  the  KPI  which  is 
defined  as  one  of  the  performance  measures  for  SC  systems.  This  may  be 
an  indication  that  most  companies  either  do  not  measure  or  researchers 
have  ignored  these  areas  for  measuring  the  performance.  However,  they 
may  have  significant  impact  on  the  overall  performance. 
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4.  From  the  perspective  of  components  of  PMs:  time  and  productivity  (46% 
and  40%  respectively  of  the  KPIs)  have  significant  weight  in  measuring 
the  performance. 

5.  Resource  utilization  and  flexibility  (35%  and  27%,  respectively  of  the 
KPIs)  have  not  been  measured  considering  the  fact  that  they  are 
intangibles  and  difficult  to  measure.  However,  they  play  a  major  role  in 
effective  management  of  SC  systems. 

6.  In  the  location  of  PMs  along  the  supply  chain;  the  performance  of 
planning  and  product  design,  supplier,  production  and  delivery  constitutes 
50%,  15%,  35%  and  12%  respectively  of  the  KPIs.  It  is  to  be  noted  that 
measuring  the  performance  related  customer  satisfaction  (27%  of  the 
KPIs)  has  not  been  given  due  consideration  in  measuring  the  performance 
of  SC.  (Gunasekaran  &  Kobu,  2007) 


The  quip  “what  gets  measured,  gets  done”  has  validity,  especially  within  supply 
chain  management.  But  the  Air  Force’s  sustainment  process  does  not  use  many  of  the 
KPIs  observations  (above)  from  civilian  industry. 


Air  Force  Sustainment  Metrics 

Air  Force  sustainment  metrics  include  both  historical  and  scorecard  (dashboard) 
metrics.  This  section  reviews  these  metrics  and  discusses  the  evolution  of  the  scorecard 
metrics. 

Historical  Metrics 

Similar  to  what  Gunasekaran  and  Kobu  presented  as  their  first  observation  from 
their  research,  the  military  also  understands  the  importance  of  internal  business  processes 
and  the  customer  play  a  significant  role  in  sustainment  operations.  Therefore,  identifying 
the  customer  is  imperative  to  what  the  Air  Force  should  use  for  KPI.  Although  the  Air 
Force  Supply  function  serves  a  variety  of  functions  at  a  tactical  level,  including 
Transportation,  Civil  Engineer  and  Services  organizations,  the  main  customer  is  the 
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Maintenance  organization.  Following  the  importance  of  this  customer,  the  Air  Force 
Logistics  Management  Agency  (AFLMA)  has  identified  four  supply-related  metrics  for 
supporting  maintenance  (AFLMA,  2001): 

1.  Issue  Effectiveness  (IE)  Rate:  This  metric  is  most  representative  of  the 
customer’s  view  of  the  support  received  from  the  supply  function.  It  provides  the 
percentage  of  customer  requirements  filled  with  assets  in  stock.  It  is  calculated 
using  the  following  equation: 

Issue  Effectiveness  = _ Issues _  x  100 

Issues  +  All  Backorders  ^ 

2.  Stockage  Effectiveness  (SE)  Rate:  This  is  similar  to  issue  effectiveness;  it 
measures  the  percentage  of  customer  requirements  filled  by  base-level  stock,  but 
only  accounts  for  items  with  an  authorized  stock  level  at  that  location. 


Stockage  Effectiveness  = _ Issues _  x  100 

Issues  +  All  Backorders  -  4W  Backorders 


(2) 


3.  Repair  Cycle  Time:  Used  primarily  as  a  local  management  indicator,  this  metric 
accounts  for  the  number  of  days  an  unserviceable  part  spends  in  the  repair  cycle. 
The  intent  is  to  minimize  this  time  to  get  unserviceable  parts  back  into  the  Air 
Force  supply  system  for  use.  It  is  comprised  of  four  components,  as  follows: 
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Repair  Cycle  Time  =  Pre-maintenance  Days  +  Repair  Days  +  Post-maintenance 
Days  -  Awaiting  Parts  (AWP)  Days  Number  of  Items 
Turned  In 


4.  Average  Repair  Cycle  Time  by  Segments:  This  metric  breaks  down  the  total 
repair  cycle  time  metric  into  its  individual  components  to  analyze  how  efficient 
each  step  is  in  the  process.  It  is  represented  as  follows: 


Total  Repair  Cycle  Time  =  Pre-Maintenance  Days  +  Repair  days  +  Post- 

Maintenance  Days 


(4) 


AFLMA  also  identifies  two  supply-focused  metrics  listed  as  Maintenance-Related 
metrics,  due  to  maintenance’s  ability  to  influence  these  metrics.  They  are: 

1.  Total  Not-Mission-Capable  for  Supply  (TNMC-S):  This  metric  identifies  the  rate 
possessed  aircraft  are  unavailable  due  to  parts.  Although  this  metric  is  primarily 
influenced  by  spare  parts  availability.  Maintenance  can  affect  it  by  consolidating 
the  supply  requirements  to  as  few  aircraft  as  possible. 

TNMCS  Rate  =  NMCS  Hours  +  NMCB  Hours  x  100 

Possessed  Hours 

( 

2.  Cannibalization  (Cann)  Rate:  This  metric  shows  the  number  of  cannibalization 
actions  taken  by  Maintenance  for  every  100  hours  the  aircraft  has  flown.  Cann 
rate  are  typically  linked  to  the  supply  issue  effectiveness  rate,  since  canns  are 
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normally  performed  when  the  supply  system  is  unable  to  issue  a  part.  The  metric 
includes  both  aircraft  and  engine  canns,  as  shown  in  the  following  equation: 


CANN  Rate  =  #  of  Acft-to-Acft  CANNs  +  #  of  Engine -to -Acft  CANNs  x  100 

Total  Sorties  Flown 


(6) 


These  6  metrics  are  highlighted  as  a  sampling  of  many  supply  chain  indicators, 
including  the  33  metrics  listed  in  the  AFLMA  Maintenance  Metrics  Handbook  (AFLMA, 
2001).  All  of  these  metrics  are  historical  metrics. 


Scorecard  and  Dashboard  Metrics 

As  previously  discussed,  the  concept  of  the  scorecard  and  dashboard  metrics  is  to 
combine  traditional,  rear-looking  metrics  with  strategic  goals  to  define  a  real-time 
operational  performance.  The  Air  Force  uses  the  scorecard  concept  in  sustainment 
operations  and  is  discussed  by  Harper  (2012)  in  recent  research  on  supply  chain  risk 
management.  Although  Harper’s  research  presents  a  simulation  model  for  introducing 
and  assessing  risk  on  inventory  within  the  supply  chain,  she  proposes  two  new  metrics, 
reviews  existing  metrics  and  discusses  aggregation  or  disaggregation  of  metrics. 
Providing  a  solid  foundation  of  the  current  metrics  structure,  Harper  presents  a  BSC 
framework,  shown  in  Figure  3,  replacing  the  four  perspectives  from  Kaplan  and  Norton 
with  Air  Force-specific  perspectives  of:  1)  Warfighter,  2)  Logistics  Process,  3)  Resource 
Planning,  and  4)  Workforce  and  Innovation. 
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Warfighter  Perspective 


Provide  optimum  responsive  logistics 
support  to  the  joint  warfighter  to  ensure: 

■  Immediately  employable  force  options 

■  A  rapidly  deployable  capability 

■  A  sustainable  total  force 


Workforce  S  Innovation  Perspective 

▲ 

Logistics  Process  Perspective 

Ensute  a  skilled  capable  workforce 

t 

Provide  effective  logistics  chain 

responsible  for  meeting  the  warfi<#iter 
logistics  support  requirements  to 
include: 

•  Introduction  of  leading  edge  advanced 
concepts 

•  Organization  adaptability 

•  Workforce  shaping 

Vision 

and 

Stiategy 

performance  and  capacity  while  reducing 
the  logistics  footprint  to  include: 

•  Reducing  cycle  time,  improving 
effectiveness,  improve  product  quality, 
leveraging  IT  improvements  and 

Commercial  Advances 

Resource  Planning  Perspective 

Ensure  affordable  logistics  suppon  through 
resources  and  choices  that  enable  effective 
joint  warfighter  capability  to  include: 

■  Accurately  forecast  and  identify  logistics 
requirements 

■  Identify  and  understand  the  risks  associated 
with  logistics  resource  allocation 

•  Improved  cost  predictability  and  constrained 
cost  growth. 


Figure  3:  Balanced  Scorecard  (Harper,  2012) 


Harper  further  assigns  10  sustainment  metrics  across  four  Air  Force  perspectives, 
as  shown  in  Table  2;  metrics  include  two  of  the  customer-focused  metrics  identified  by 
AFLMA,  IE  and  SE. 


Table  2:  Balanced  Scorecard  Framework  (Harper,  2012) 


Warfighter 

Perspective 

Resource  Planning 
Perspective 

Logistics  Process 
Perspective 

Workforce  & 
Innovation 
Perspective 

MICAP  Hours 

NOR 

AA 

AA 

CWT 

IE 

MICAP  Hours 

TRY 

Perfect  Order 
Fulfillment 

SE 

MICAP  Incidents 

TRY 

Backorders 
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Although  the  balanced  scorecard  provides  real-time  metrics  that  are  useful  to  the 
Air  Force,  they  do  have  major  challenges.  Five  main  criticisms  of  the  scorecard  are 
presented  by  the  DeGroote  School  of  Business  (2008): 

1.  Linkage  to  strategy:  Only  5%  of  the  workforce  understands  their 
company  strategy.  Only  25%  of  managers  have  incentives  linked  to 
strategy.  60%  of  organizations  don’t  link  budgets  to  strategy.  86%  of 
executive  teams  spend  less  than  one  hour  per  month  discussing  strategy. 

2.  Choice  of  Information:  The  choice  of  what  measures  to  use  is  often 
based  on  the  information  that  is  most  easily  obtainable,  rather  than  the 
most  useful. 

3.  Scorecard  Revision:  Questioning  the  assumptions  held  about  the 
organization’s  strategy,  and  of  the  linkages  and  measures  of  the  Balanced 
Scorecard,  particularly  when  the  actual  results  differ  from  the  expected 
results  is  essential.  If  the  strategy  is  found  to  be  lacking  the  organization 
will  need  to  refine  it  and  may  consequently  revise  some  or  all  of  the 
measures  on  the  balanced  scorecard. 

4.  Lagging  Metrics:  The  measurements  are  referred  to  as  lagging 
indicators  and  they  dominate  most  performance  measurement  systems. 
About  70%  of  all  measurements  tend  to  fall  into  this  category.  One  of  the 
major  challenges  in  building  your  balanced  scorecard  is  to  keep  the 
number  of  measurements  to  a  manageable  few. 

5.  Apples  and  Oranges:  Benchmarking  works  well  when  the  process 
being  benchmarked  is  essentially  the  same  at  the  multiple  units  (either 
internal  or  external)  participating  in  the  exercise.  Benchmarking  is  not 
informative  when  it  is  used  to  compare  fundamentally  different  processes 
or  products.  Furthermore,  it  is  not  effective  when  the  companies  being 
compared  work  from  significantly  different  strategies.  (DeGroote  School 
of  Business,  2008) 


Requirements  Management  System  (D200) 

The  Requirements  Management  System  (RMS),  commonly  called  D200, 
computes  worldwide  spare  parts  requirements  for  Air  Force  managed  assets.  The  system 
was  originally  designed  with  nine  subsystems  to  cover  the  breadth  of  the  supply 
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processes,  but  one  was  never  deployed  and  two  have  since  been  deactivated.  There 
primary  subsystem  is  the  D200A  and  three  supporting  subsystems  that  feed  into  D200A. 
These  four  main  subsystems  are: 


D200A:  Secondary  Item  Requirements  System  (SIRS):  computers  buy, 
repair,  termination  and  excess  quantities  for  consumable  and  recoverable 
supply  items. 

D200E:  Requirements  Item  Identification  (RIID):  receives  weekly  feeds 
from  the  D043  Cataloguing  System.  Provides  stock  list  changes,  stock 
numbers,  substitutability  structures,  lead  time  and  cost  data  from  the 
acquisition  system. 

D200F:  Application,  Programs,  Indenture  (API):  provides  D200A  with  past 
and  future  data  on  operations  and  maintenance  activities  which  create  a 
demand  for  repair  parts.  Information  includes  flying  hours,  inventory, 
ammunition,  sorties  flown,  programmed  depot  maintenance  and  engine 
overhaul. 

D200N:  Central  Secondary  Item  Stratification  (CSIS):  stratifies  D200A  data 
into  anticipated  fiscal  year  expenditures. 

Additionally  the  Logistics  Management  Data  Bank  (D043)  is  an  extension  of 

D200N.  Once  the  budget  is  pushed  from  D200N  into  D043,  it  makes  final  adjustments  to 

the  supply  budget  prior  to  submission  to  Congress.  The  Requirements  Determination 

System  is  mapped  out  below  in  Figure  2. 
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Figure  4:  Requirements  Determination  System  Process  (401  SCMS/GUMD,  2011) 


Most  users  are  familiar  with  the  D200A  subsystem  of  RMS,  as  data  feeds  into  this 
system  and  provides  a  variety  of  analytical  results.  The  plethora  of  input  and  output  are 


below  in  Figure  5,  and  illustrate  the  complexity  of  data  mining  and  analysis  required. 


Figure  5:  D200A  Inputs  and  Outputs  (401  SCMS/GUMD,  2011) 
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The  Aircraft  Availability  Model  (AAM)  is  an  optimization  program  embedded 
with  D200A  and  is  used  to  compute  safety  levels  of  stock.  For  aircraft  items,  AAM 
optimizes  inventory  (stock)  level  requirements,  subject  to  aircraft  performance  goals  and 
cost  thresholds.  For  non-aircraft  items,  AAM  optimizes  inventory  (stock)  level 
requirements,  subject  to  expected  backorders  (i.e.,  Customer  Wait  Time)  and  fill-rate  and 
cost  thresholds.  AAM  provides  two  main  output  for  safety  levels  for  base-level  stock  and 
depot-level  stock,  as  shown  in  the  following  two  equations: 

Base  Safety  Level  =  Base  Stock  Level  -  Base  Pipeline  ^ 

Depot  Safety  Level  =  Depot  Stock  Level  -  Depot  Pipeline  . 


The  RMS,  using  D200A  and  the  Aircraft  Availability  Model,  has  a  forward- 
looking  capability,  which  in  turn  provides  recommendations  for  optimal  levels  of  safety 
stock.  But  currently,  the  system  isn’t  being  used  at  its  fullest  capability.  At  the  tactical 
and  operational  levels,  it  is  used  for  real-time  decision  making,  and  for  acquisition  at  the 
strategic  level.  When  asked  why  this  program  wasn’t  used  in  a  forward-looking  capacity, 
multiple  leaders  within  the  supply  chain  stated  a  variation  of  “garbage  in,  garbage  out.” 

Air  Force  Sustainment  Metrics  Initiatives 

In  2007,  the  Government  Accountability  Office  (GAO)  published  their  report 
“DoD’s  High  Risk  Areas:  Progress  Made  Implementing  Supply  Chain  Management 
Recommendations,  but  Full  Extent  of  Improvement  Unknown.”  It  was  a  follow-up  report 
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from  2001  to  assess  how  well  the  DoD  had  implemented  previous  recommendations  on 
improving  the  supply  chain.  The  main  recommendation  from  the  2007  report  was  that 
“DOD  complete  its  logistics  strategy  and  develop  and  implement  outcome  focused 
performance  metrics  and  cost  metrics  for  supply  chain  management”  (GAO,  2007) 

The  Air  Force  Sustainment  Center  is  responsible  for  implementing  this 
recommendation  within  the  Air  Force,  as  their  mission  is  Sustain  Weapon  System 
Readiness  to  generate  Airpower  for  America.  The  center  provides  expeditionary 
capabilities  to  the  warfighter  through  depot  maintenance,  supply  chain  management  and 
installation  support  (AFSC,  2012).  This  section  covers  the  current  research  and 
initiatives  ongoing  in  the  Air  Force  logistics  community  applicable  to  this  research. 

Cascading  Air  Force  Sustainment  Metrics 
(635  SCOW,  2012) 

In  December  2012,  personnel  from  the  Supply  Chain  Operations  Wings  (SCOW) 
from  both  the  Mobility  and  Combat  Air  Forces  met  for  an  Integrated  Process  Team  (IPT) 
to  map  out  the  current  Air  Force  Supply  Metrics  to  show  how  they  affect  overall  Fleet 
Availability  (FA).  FA,  also  known  as  aircraft  availability  (AA),  is  the  overall  indicator 
used  by  the  Air  Force  to  determine  the  health  of  the  fleet  (AFLMA,  2001).  The  SCOWs 
were  tasked  by  the  Air  Force  Sustainment  Center  (AFSC  with  an  intended  interim 
operational  capability  (IOC)  of  January  2013,  and  full  operational  capability  (FOC)  in 
April  2013.  Although  the  IPT  members  did  not  feel  they  could  attain  the  prescribed 
IOC/FOC  timeline,  they  were  successful  in  mapping  out  a  hierarchy  of  metrics,  as  shown 
in  Appendix  A. 
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As  can  be  seen  in  the  figure  in  Appendix  A,  the  main  supply  metrics  the  Air  Force 
uses  affect  AA  through  different  pipelines  and  often  affect  each  other.  The  legend  also 
depicts  which  metrics  are  automated,  readily  available,  manually  collected,  etc.  The 
Cascading  Metrics  team  also  has  recommended  two  new  metrics,  Expected  Delivery  Day 
Accuracy  and  Alternate  Resolution  options  when  for  stock  outs.  Unfortunately,  it 
accurately  depicts  the  complexity  in  gathering  data  within  the  Air  Force  Supply  System, 
leaving  much  to  be  desired.  As  of  the  conclusion  of  this  research,  this  initiative  is  still 
ongoing. 


Enterprise  Supply  Chain  Analysis,  Planning  and  Execution  ( ESCAPE ) 
(HQ  AFMC/A4N,  7  Dec  12) 

There  is  not  a  single,  authoritative  source  for  (AF)  supply  chain  planning 
data,  forcing  a  heavy  reliance  on  multiple  automated  systems,  providing 
conflicting  data.  These  disparate  systems  are  found  throughout  the  supply 
chain  and  may  be  unique  to  their  functional  stovepipe  or  part  of  a  legacy 
suite  of  applications.  They  provide  data  dissimilarities,  lack  data  integrity 
and  lack  of  discipline  in  providing  supply  chain  data.  Current  IT  systems 
were  originally  designed  in  the  1960s-1970s,  and  upgraded  to  newer 
technology  when  possible;  many  of  the  processes  remain  the  same  as 
originally  designed.  (HQ  AFMC/A4N,  7  Dec  12) 


The  Air  Force  Materiel  Command  (AFMC)  Directorate  of  Fogistics  (A4)  is 
working  toward  a  supply  chain  capability  called  the  Enterprise  Supply  Chain  Analysis, 
Planning  and  Execution  (ESCAPE)  system.  The  system  is  a  derivative  of  the 
Expeditionary  Combat  Support  System  (ECSS),  an  integrated  technology  system  that  was 
being  built  for  the  Air  Force,  but  was  recently  terminated.  ESCAPE  is  being  developed 
to  provide  a  “comprehensive,  integrated,  accurate  and  timely  supply  chain  planning” 
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system.  This  will  be  accomplished  via  five  modules:  Demand  Planning,  Inventory 

Management,  Supply  Planning,  Exception  Management  and  Analytics.  The  Analytics 

section,  of  note,  will  enable  a  capability  in  data  stratifications  and  metrics  dash-boarding 

not  currently  available.  As  noted  by  AFMC/A4N: 

The  Air  Force  supply  chain  planning  functions  currently  lack  the  ability  to 
optimize  resources  across  the  enterprise  and  have  no  standard 
management  processes  to  integrate... (or)  optimize  weapon  system 
availability... which  leads  to  inaccurate,  inconsistent  and  un-timely 
information  required  for  effective  decision  making.  (2012) 

The  result,  as  the  report  goes  on  to  say,  is  “un- supportable  plans  that  cause 

increased  MICAPs  and  longer  maintenance  flow  days.”  (2012).  Among  the  many 

business  capabilities  ESCAPE  intends  to  provide,  it  aims  to  develop  Enterprise  Metrics 

that  are  transparent,  standardized,  timely  and  cost-effective.  Similar  to  the  work  being 

done  on  Cascading  Metrics  by  the  SCOW,  the  ESCAPE  team  has  identified  a  need  for  a 

hierarchy  of  interrelated  metrics.  The  intended  outcome  is  to  develop  a  supply  chain 

planning  capability  to  better  optimize  and  align  scarce  resources. 

ESCAPE  is  being  implemented  using  the  six-step  Service  Development  and 

Deliverable  Process  (SDDP).  SDDP  was  designed  by  the  Air  Force  in  2010  to  define 

how  the  Air  Force  will  use  business  process  reengineering  to  shape  and  implement  new 

solutions  (USAF,  2011).  As  of  the  conclusion  of  this  research,  the  ESCAPE  system 

completed  the  first  step  of  the  SDDP,  Identification  of  Capability  Requirements,  and  will 

soon  be  entering  into  the  next  stages  (Bumworth,  2013). 
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National  Stock  Number  (NSN)  Management  Initiative 

(Baird,  2012) 

Sustainment  operations  in  the  Air  Force  have  undergone  many  changes  over  the 
past  decade.  Prior  to  2000,  sustainment  operations  were  managed  by  the  base-level 
Supply  Squadrons.  Each  base  had  their  own  supply  personnel  to  manage  inventory, 
MICAP,  stock  control,  equipment  levels,  etc.,  for  their  primary  assigned  aircraft  at  their 
base.  Although  most  aircraft  fleets  were  spread  across  multiple  bases  and  usually  more 
than  one  Major  Command  (MAJCOM),  sustaining  the  fleet  was  not  primarily  managed  at 
a  strategic  level. 

In  2000,  the  Air  Force  stood  up  Regional  Supply  Squadrons  (RSS)  at  five 
MAJCOMs  to  centralize  sustainment  operations  within  the  MAJCOM.  The  five 
MAJCOM  RSSs  became  responsible  for  specific  functions  formally  owned  by  their  base- 
level  supply  squadrons,  to  include  stock  control,  MICAP  and  customer  service.  The 
personnel  and  processes  were  relocated  from  the  base-level  to  the  centralized  RSS,  but 
there  was  no  process  change  in  how  to  manage  the  inventory  for  each  base.  Early 
iterations  of  a  need  for  the  NSN  Management  Initiative  emerged  as  the  RSSs  became 
fully  operational  and  began  “stepping  on  each  other’s  toes,”  as  some  bases  had  multiple 
MAJCOM’s  aircraft  assigned. 

Further  realignments  have  occurred  since  then.  The  five  RSSs  merged  into  two 
RSSs,  one  for  the  Mobility  Air  Forces  (MAF)  and  one  for  the  Combat  Air  Forces  (CAF) 
aircraft,  but  had  still  not  worked  out  some  of  the  emerging  issues  due  to  how  the  process 
was  managed.  Then  the  Air  Force  Global  Fogistics  Support  Center  was  established,  but 
only  remained  active  for  a  few  years.  The  latest  change  of  structure  came  in  June  2010, 
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when  the  635th  Supply  Chain  Operations  Wing  (SCOW)  was  established  at  Scott  AFB 
(Robertson,  2010). 

Under  the  SCOW  structure,  there  are  two  groups  (SCOG)  and  four  squadrons 
(SCOS).  The  SCOGs  are  located  at  Scott  AFB  (managing  MAF  aircraft)  and  Langley 
AFB  (managing  CAF  aircraft).  Although  there  is  a  new  structure,  the  same  problems 
still  remain.  For  example,  Kadena  AB  Japan  has  five  weapon  systems  which  are 
managed  across  all  four  of  the  squadrons.  Therefore,  supply  actions  that  arise  for 
common  stock  number’s  originating  from  Kadena  AB  will  potentially  be  worked  by 
multiple  squadrons. 

The  intent  of  the  NSN  Management  Initiative  is  to  have  one  owner  of  a  NSN  at 
retail  level  to  work  all  processes  associated  with  that  NSN,  such  as  requirements, 
cancellations,  backorders,  etc.  Of  the  more  than  3.5  million  NSNs  supporting  Air  Force 
aircraft,  only  approximately  678,000  are  unique  NSNs  (meaning  only  used  on  one 
aircraft).  This  causes  significant  overlap;  since  almost  3  million  NSNs  are  used  on  more 
than  one  aircraft,  managing  by  aircraft  type  means  you  have  multiple  owners  of  those 
NSN,  causing  duplication,  additional  work  and  often  rework. 

This  program  hopes  to  extend  the  retail-level  NSN  management  beyond  just  the 
Air  Force  managed  supply  items  and  include  contract-managed  NSNs  as  well.  Recently, 
a  supply  part  managed  by  Lockheed  Martin  was  due  for  contract  renewal,  which  is 
outside  of  the  purview  of  the  weapon  system  managers  (WSM).  This  particular  contract 
was  having  issues  and  there  was  going  to  be  a  gap  in  supply  support  during  this  delay. 
Fortunately,  a  Lockheed  Martin  representative  contacted  the  retail  level  WSM  to  notify 
of  the  issues  and  the  SCOS  was  able  to  plot  a  course  of  action  during  the  60  to  90-day 
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gap  in  support.  Leaning  forward,  the  SCOS  increased  their  stock  level  to  accommodate 
demand  during  the  gap  until  support  was  brought  back  online.  But  this  raised  the 
question,  who  should  be  monitoring  these  contracts  within  the  supply  chain?  (Gehrich, 
2013) 

In  July  2012,  the  NSN  Management  Initiative  began  its  testing  phase  and 
identified  constraints  related  to  the  reports  it  would  be  using.  According  to  Mr.  Gerry 
Baird,  436  SCOS  Analyst,  the  project  is  currently  halted  to  remedy  these  problems  and 
will  resume  testing  in  late  spring  2013. 

Summary 

This  literature  review  presented  background  information  on  the  three  types  of 
metrics:  historical  metrics,  dashboard/scorecard  metrics  and  forward-looking  metrics. 
The  Air  Force  is  primarily  using  historical  and  scorecard  metrics  for  sustainment 
operations.  A  recent  interest  in  developing  a  new  way  to  use  our  metrics  has  arisen  in  the 
Air  Force  over  the  past  few  years  and  has  introduced  many  initiatives,  including  the 
Cascading  Metrics,  ESCAPE  and  NSN  Management  Initiatives.  But  none  of  these 
initiatives  have  been  able  to  produce  a  forward-looking  metric. 

Therefore,  this  literature  review  has  answered  the  first  two  investigative 
questions:  1)  What  metrics  are  currently  used  for  sustainment  operations  in  the  Air 
Force?  and  2)  What  are  forward-looking  metrics?  The  remaining  two  investigative 
questions  will  be  answered  in  the  following  methodology  and  analysis  portions  of  this 
research. 
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III.  Methodology 

Chapter  Overview 

The  intent  of  this  research  is  to  develop  a  forward-looking  metric  that  can  be  used 
to  determine  how  to  change  stock  levels  to  reduce  MICAP  hours  within  the  Air  Force 
supply  system.  An  example  is  provided  using  C-5  fleet  data. 

Scope  and  Data  Description 

A  data  pull  was  conducted  from  the  RMS,  D-200A  module,  providing  data  on  all 
C-5  MICAPs  from  October  2009  to  Sep  2012.  The  data  set  included  79  different  fields, 
with  over  30,500  lines  of  data,  which  will  further  be  referred  to  as  the  main  data  set.  As 
of  the  June  2012  D200A  Summary  Computation,  there  were  2,862  primary  component 
NSNs  associated  with  the  C005A/B/C/M  aircraft.  However,  there  were  7,851  total  NSNs 
that  had  a  MICAP  action  associated  with  it  over  the  given  timeframe.  Only  three  of  the 
fields  from  this  main  data  set  were  used:  date,  NSN,  and  MTD  MICAP  Hrs.  The 
variables  and  NSNs  used  in  each  of  the  optimization  models  are  listed  in  Appendices  B 
through  F. 

Model  Development 

To  minimize  the  scope  for  the  first  model  (further  known  as  Model  1),  only  the 
primary  C-5  Structural  Component  NSNs  were  used,  which  included  134  NSNs.  This 
did  not  include  Interchangeable  &  Substitution  Group  (I&SG)  parts.  Narrowing  down 
the  main  data  set  to  just  the  structural  NSNs  reduced  the  MICAP  data  to  2,075  lines,  with 
only  123  NSNs  having  a  MICAP  action  during  the  given  timeframe.  Each  of  these  123 
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NSNs  were  labeled  with  an  X  variable  (X001,  X002,  X003  ...  X123)  for  simplification. 
A  pivot  table  was  built  for  a  cursory  analysis  of  the  data  and  two  spreadsheets  were  built 
from  the  pivot  table.  One  spreadsheet  shows  the  number  of  MICAPs  in  the  system  each 
month,  by  NSN.  The  second  spreadsheet  shows  the  total  amount  of  MICAP  hours  each 
NSN  accrued  each  month.  To  simplify  the  set  into  readily  useable  data,  an  assumption 
was  made  that  all  MICAPs  were  satisfied  within  the  same  month  they  went  MICAP.  If  a 
part  was  MICAP  and  counted  in  two  or  more  sequential  months,  it  appears  as  separate 
MICAPs. 

The  two  spreadsheets  were  further  divided  into  individual  fiscal  year  (FY)  sheets, 
FY 10,  FY  1 1  and  FY12.  All  subsequent  analysis  for  Model  1  was  performed  on  FY  10 
data.  NSNs  with  no  MICAP  Hours  listed  were  deleted  from  the  sample.  Reasons  some 
of  the  NSNs  had  zero  MICAP  hours  is  likely  due  to  either  the  NSN  did  not  have  any 
MICAPs  during  FY  10,  but  had  MICAPs  in  the  out  years,  or  the  MICAP  action  was 
satisfied  before  it  could  accrue  one  hour  of  time.  This  reduced  the  FY  10  set  to  92  NSNs. 
Next,  the  mean  of  MICAP  hours  (hr)  and  number  of  MICAPs  (#)  for  each  NSN  were 
obtained  using  the  following  equations: 


/r(/ir)  - 


Annual  MICAP  Hours 
Annual  Number  of  MICAPs 


=  h 


(9) 


p(#)  = 


Annual  Number  of  MICAPs 
12 


=  d 


(10) 


An  optimization  linear  program  was  developed  using  Excel  Solver,  with  the  following 
formulation: 

Min  cT(x) 
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Subject  to:  hT(x)  >=  m 
I(x)  <=  d 


(11) 


Variables  represented:  c  is  the  “cost”  of  each  NSN,  which  is  represented  as  1  in 
this  model;  T  is  the  transpose  of  a  matrix;  x  is  the  dependent  variable  NSN;  h  is  the  mean 
hours  for  each  MICAP  NSN,  as  shown  in  Equation  7;  m  is  the  number  of  MICAP  hours 
that  should  decrease  each  month;  I  is  an  identity  matrix  for  each  NSN;  and  d  is  the  mean 
monthly  MICAP  requirement  (demand)  for  each  NSN,  as  shown  in  Equation  8.  The 
objective  function  in  Model  1  is  insignificant,  as  the  objective  is  to  define  which 
variables  should  be  put  into  stock  to  decrease  the  monthly  MICAP  hours. 

Based  on  the  data  set  for  Model  1,  the  mean  of  the  monthly  MICAP  hours  was 
14,000.5  hours.  Therefore,  “m”  cannot  be  any  larger  than  14,000.5,  as  you  cannot 
decrease  the  hours  more  than  are  available  to  decrease.  Next,  it  was  determined  by  how 
much  to  reduce  the  MICAP  hours  (m);  Model  1  used  a  10%  decrease  of  the  maximum 
MICAP  hours,  thus  decreasing  it  by  1,400.05  hours.  Model  1  provided  two  variables 
with  applicable  amounts  by  which  to  decrease  the  overall  MICAP  hours,  as  shown  in 
Table  4. 


Table  3:  Model  1  Output  (FY10) 


Variable 

NSN 

Amount 
(per  month) 

X044 

1560012611313 

0.41667 

X048 

1560014372544 

2.34381 
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Robust  Variable  Selection  for  Determining  Candidate  Forward  Looking  Metrics 

Unfortunately,  variable  X044  is  less  than  a  whole  number,  meaning  there  is  not  a 
stable  MICAP  requirement  throughout  the  year  for  the  NSN  to  evoke  at  least  one 
additional  asset  on  hand.  This  contradicts  the  desired  result  of  determining  which  NSNs 
should  be  increased  by  at  least  one  to  reduce  MICAP  hours.  Therefore,  a  second  model 
(Model  2)  with  business  rules  is  required  to  provide  a  robust  data  set  to  determine 
candidate  variables.  This  robust  variable  selection  for  Model  2  includes  the  following 
business  rules: 

1.  Variable  Median  MICAP  Demand  (d)  >  1 

2.  Mean  MICAP  Hours  per  MICAP  action  (h)  >  24  hours 

The  first  business  rule  divides  the  MICAP  hours  into  two  categories:  Recurring 
MICAP  hours  and  Non-recurring  MICAP  hours.  Recurring  MICAP  hours  have  a  median 
demand  greater  than  or  equal  to  one,  which  indicates  a  stable  MICAP  requirement 
throughout  the  fiscal  year.  Non-recurring  MICAP  hours  had  a  median  MICAP 
requirement  less  than  one,  indicating  an  unstable  requirement  or  random  occurrence 
throughout  the  fiscal  year.  This  business  rule  changes  “d”  from  the  mean  found  in 
equation  8  to  the  median. 

The  second  business  rule  eliminates  the  recurring  MICAPs  that  were  satisfied  in 
less  than  24  hours,  as  determined  by  equation  7.  MICAPs  delivered  on  average  less  than 
24  hours  are  not  useful  to  this  model  since  it  assesses  AF-wide  MICAP  requirements  and 
not  specific  placement  of  parts.  These  NSNs  are  likely  already  in  the  supply  system  and 
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able  to  be  satisfied  via  lateral  support  or  cannibalization  action.  These  NSNs  are  further 
referred  to  as  “In-Inventory”  MICAPs. 

Applying  the  robust  data  selection  business  rules  to  the  FY 10  data  set  reduced  the 
variables  from  92  variables  in  Model  1  to  17  variables  for  Model  2.  An  optimization 
linear  program  was  developed  using  the  same  parameters  from  equation  9  used  in  Model 
1,  with  the  robust  data  set. 

Final  Model  Development 

Based  on  the  data  set  for  Model  2,  the  median  of  the  monthly  MICAP  hours  was 
9351 .79  hours.  Therefore,  “m”  in  the  constraints  of  our  model  formulation  cannot  be  any 
larger  than  9351.79.  Following  the  same  methodology  in  Model  1,  Model  2  used  a  10% 
decrease  of  the  maximum  MICAP  hours,  thus  decreasing  it  by  935.179  hours.  Model  2 
provided  one  variable  with  applicable  amount  that  will  enable  the  desired  MICAP  hour 
decrease  (m),  as  shown  in  Table  5. 


Table  4:  Model  2  Output  (FY10) 


Variable 

NSN 

Amount 
(per  month) 

X048 

1560014372544 

1.879211173 

This  variable  was  also  present  in  the  Model  1  output  set,  but  since  the  population 
size  of  the  data  set  was  so  small  (only  17  variables),  it  was  unclear  if  the  new  robust  data 
set  had  provided  the  intended  solution.  Therefore,  Model  2.1  was  developed  using  the 
same  program  as  Model  2,  but  with  all  C-5  NSNs  (Models  1  and  2  had  limited  the  data 
set  to  only  structural  NSNs). 
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Model  2.1  (FY10)  had  78  NSN  variables  in  the  model  and  optimized  five 
variables,  as  shown  below: 


Table  5:  Model  2.1  Output  (FY10) 


Variable 

NSN 

Amount 
(per  month) 

X1058 

1560008718185 

1 

XI 429 

1560014372544 

3.75 

X1432 

1560014372548 

4.29769 

X1441 

1560014536213 

1.58333 

X2859 

4510004101062 

6.25 

Model  2.1  (FYll)  had  52  NSN  variables  in  the  model  and  optimized  four 
variables,  as  shown  below: 


Table  6:  Model  2.1  Output  (FYll) 


Variable 

NSN 

Amount 
(per  month) 

X1211 

1560011160447 

1 

X1432 

1560014372548 

2 

X2001 

1680001033526 

1 

X5294 

5330015771850 

0.748162256 

Model  2.1  (FY12)  had  40  NSN  variables  in  the  model  and  optimized  five 
variables,  as  shown  below: 


Table  7:  Model  2.1  Output  (FYll) 


Variable 


NSN 


Amount 
(per  month) 
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X1250 

1560011795638 

1 

XI 429 

1560014372544 

1.789481719 

X1432 

1560014372548 

1.5 

X1520 

1560016030195 

1 

X2230 

1680011775069 

1.5 

Summary 

In  summary,  the  Forward-Looking  Metric  development  process  is  illustrated  in 
Figure  6. 


Figure  6:  Development  of  Forward-Looking  Metric 

Utilizing  D200A  data,  the  Monthly  MICAP  Hours  data  for  all  C-5  NSNs  from 
October  2009  to  September  2012  was  extracted  into  an  Excel  spreadsheet.  Each  of  the 
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7,85 1  NSNs  with  MICAP  actions  during  that  timeframe  were  assigned  an  “X”  variable 
(X0001,  X0002  . . .  X785 1).  Using  the  Pivot  Table  function  of  Excel,  two  queries  were 
performed  to  1)  count  the  number  of  MICAPs  that  occurred,  by  month,  for  each  NSN  and 
2)  sum  the  MICAP  Hours  accrued,  by  month,  for  each  NSN.  This  data  was  further 
separated  into  individual  spreadsheets  for  FY10,  1 1  and  12.  Each  year’s  worth  of  data 
then  underwent  the  Robust  Variable  Selection  Process,  summarized  in  Table  8. 


Table  8:  Robust  Variable  Selection:  Business  Rules 


MICAP  Category 

Rule  1:  Monthly  Median 
MICAP  Demand 

Rule  2:  Monthly  Mean  Hours 
per  MICAP  Action 

Non-Recurring 

<  1 

N/A 

Recurring 

>=  1 

>  =  24  hours 

In-Inventory 

N/A 

<  24  hours 

The  Robust  Variable  Selection  process  subdivided  the  data  sets  into  three 
categories  of  MICAPs:  Non-Recurring  MICAPs,  Recurring  MICAPs  and  In-Inventory 
MICAPs.  With  only  the  Recurring  MICAP  data  set,  an  optimization  linear  program  was 
developed  to  provide  the  forward-looking  metric  for  change  in  stock  levels  as  a  result  of 
Recurring  MICAP  parts. 

Model  2.1,  with  the  robust  data  selection  business  rules,  provided  a  sound 
optimization  program  to  developing  a  forward  looking  metric  for  changing  stock  levels  to 
reduce  MICAP  hours.  Further  analysis  is  done  in  the  next  section,  which  illustrates  a 
refinement  of  Model  2.1  via  Models  3  and  4. 
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IV.  Analysis  and  Results 

Chapter  Overview 

This  section  presents  the  results  from  Models  2.1  for  FY  10,  11  and  12  and 
analyzes  the  data  into  useable  information.  Further,  two  additional  models  are  proposed 
for  real-world  application.  Model  3  presents  an  integer  format  of  Model  2. 1  for  whole 
number  stock  level  changes.  Model  4  incorporates  unit  price  to  gain  not  only 
effectiveness  but  also  efficiency  within  the  forward  looking  metric.  Ultimately,  Model  4 
presents  the  best  results  for  a  forward-looking  metric  and  can  be  utilized  across  the  other 
AF  weapon  systems  for  sustainment  operations. 

Analysis  of  Model  2.1 

Model  2.1  provides  a  sound  optimization  program  for  a  forward-looking  metric; 
three  linear  programs  were  run  using  FY  10,  FY  1 1  and  FY  12  data  individually.  Using  a 
25%  reduction  of  Recurring  MICAP  hours  based  on  the  average  Recurring  MICAP  hours 
per  month,  the  following  results  were  produced  from  Model  2.1: 


Table  9:  Model  2.1  Results 


FY 

Variables 

Mean  MICAP 

Hrs 

25%  MICAP  Hr 

Reduction 

Number  of  NSNs 

Optimized 

FY10 

78 

34,212.3 

8553.075 

5 

FY11 

52 

10,056.1 

2514.025 

4 

FY12 

40 

13,491.4 

3372.85 

5 
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The  number  of  variables  decreased  in  each  successive  fiscal  year  of  Model  2.1. 
This  represents  that  different  NSNs  were  recurring  MICAPs  over  the  three-year  period, 
likely  due  to  an  increase  in  stock  levels  for  the  recurring  MICAP  NSNs  to  prevent 
MICAPs  in  the  out-years.  There  were  140  different  NSNs  used  across  the  three  FY 
models;  seven  NSNs  were  used  in  all  three  models  and  16  NSNs  were  in  two  models. 
Further,  there  was  only  one  NSN  (X1432)  optimized  in  all  three  models  and  one  NSN 
(XI 429)  optimized  in  two  of  the  three  models.  Since  the  variables  changed  significantly 
from  year  to  year,  it  was  decided  to  not  combine  the  three  FYs  into  one  program,  but  to 
look  across  the  three  FYs  for  the  most  consistent  Recurring  MICAP  Hours  data 
throughout  the  year.  The  Recurring  MICAP  Hours  data  was  plotted  in  the  following 
three  figures,  using  the  cumulative  MICAP  hours  for  each  month  in  the  FY. 

The  cumulative  MICAP  hours  for  FY  10  ranged  from  60,722  hours  in  October 
2009  to  16,632  hours  in  September  2010.  The  delta  was  44,090  hours,  which  is  almost 
triple  the  total  hours  at  the  end  of  the  FY.  MICAP  hours  had  a  substantial  downward 
trend  throughout  the  year. 
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FY10  Recurring  MICAP  Hours 
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Figure  7:  FY10  Recurring  MICAP  Flours 


The  cumulative  MICAP  hours  for  FY 1 1  ranged  from  17,672  hours  in  January 
201 1  to  5,528  hours  in  September  2011.  The  delta  was  12,144  hours.  MICAP  hours  had 
a  downward  trend  for  a  majority  of  the  year. 


FY11  Recurring  MICAP  Hours 


Figure  8:  FY11  Recurring  MICAP  Hours 


The  cumulative  MICAP  hours  for  FY12  ranged  from  6,241  hours  to  22,719  hours, 
with  a  delta  of  16,478  hours.  Unlike  the  previous  two  FYs,  FY12  had  a  slight  upward 
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trend  throughout  the  year,  and  the  beginning  and  ending  values  were  7,261  and  8,624, 
respectively.  Although  FY 1 1  had  the  smallest  range  in  hours,  FY12  had  the  most 
consistent  data. 


FY12  Recurring  MICAP  Hours 


Figure  9:  FY12  Recurring  MICAP  Hours 


Charting  out  the  FY12  Non-Recurring  MICAP  hours  and  In- Inventory  MICAP 
hours  showed  similar  trend  consistency  throughout  the  year,  as  shown  in  the  following 
two  figures.  As  budgetary  constraints  become  more  prominent,  use  of  this  forward 
looking  metric  approach  will  become  critical  in  more  efficiently  managing  which  NSNs 
to  impact  improved  performance  of  the  supply  chain. 
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Figure  10:  FY12  Non-Recurring  MICAP  Hours 


FY12  In-Inventory  MICAP  Hours 


Figure  11:  FY12  In-Inventory  MICAP  Hours 


Since  the  FY 12  data  was  the  most  consistent  across  the  entire  FY  for  Recurring 
MICAP  Hours,  and  exhibited  similar  consistency  in  the  Non-Recurring  and  In-Inventory 
Hours,  it  was  determined  to  use  only  FY12  data  for  further  analysis.  This  data  is  probably 
more  consistent  with  the  current  operations  of  the  AF  supply  chain  since  contingency 
funding  is  diminishing. 
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Development  and  Analysis  of  Model  3 

The  most  productive  model  is  the  FY12  Model  2.1  because  of  low  variability  in 
difference  across  the  FY  for  a  monthly  median.  A  monthly  median  will  allow  the  data  to 
be  normalized  for  a  consistent  output.  However,  a  downfall  in  Model  2.1  still  exists  in 
that  it  provides  portions  of  a  NSN  for  optimization.  Since  portions  of  a  NSN  can  not  be 
obtained  in  the  real  world,  but  only  whole  parts,  Model  3  was  developed  as  an  Integer 
Linear  Program  of  Model  2.1.  Running  Model  3  provided  the  following  results,  as 
shown  in  Table  10. 


Table  10:  Model  3  Output  (FY12) 


Variable 

NSN 

Amount 
(per  month) 

X1250 

1560011795638 

1 

XI 429 

1560014372544 

3 

X1432 

1560014372548 

1 

X1520 

1560016030195 

1 

X2230 

1680011775069 

1 

TOTAL 

7 

Comparing  the  output  of  Model  3  with  Model  2.1  (FY  12)  shown  in  Table  8 
reveals  that  the  output  variables  did  not  change,  but  the  amount  does  reallocate  the 
distribution  to  obtain  a  whole  number,  which  is  useable  in  the  real  world.  Therefore, 
although  Model  2.1  (FY12)  provides  the  optimal  solution  to  reduce  the  MICAP  hours, 
Model  3  provides  the  optimal  real-world  solution  since  partial  NSNs  cannot  be  obtained. 
Model  3  is  a  more  useable  version  of  Model  2. 1 
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Development  and  Analysis  of  Model  4 

The  final  model  developed  for  optimizing  MICAP  hours  was  to  introduce  cost 
into  the  program.  Models  1  through  3  had  used  a  cost  of  “1”  for  all  NSNs  to  demonstrate 
an  equal  relationship  between  all  NSNs  within  the  program.  This  was  done  to  identify 
the  most  effective  means  of  optimizing  MICAP  hours  by  removing  cost  from  the 
equation.  In  the  absence  of  cost  being  the  determining  factor,  the  program  defaulted  to 
optimizing  based  on  need,  or  highest  MICAP  hours.  It  is  logically  followed  that  the 
NSNs  with  highest  MICAP  hours  (longest  time  to  satisfy  the  requirement)  means  the 
aircraft  is  down  for  parts  a  greater  amount  of  time,  likely  leading  to  an  increased  TNMC- 
S  and  AA  rates.  (Note:  since  TNMC-S  and  AA  rates  are  determined  by  the  number  of 
aircraft  available,  and  one  aircraft  could  have  multiple  MICAPs  against  it,  there  is  not  a 
direct  correlation  between  MICAP  hours  and  TNMC-S  or  AA  rates;  hence,  there  is  only  a 
likely  probability  it  would  lead  to  an  increased  rate.  However,  researching  the  link 
between  these  metrics  would  be  productive  follow-on  research.)  Therefore,  Model  3 
provides  the  most  effective  optimal  solution  for  a  forward-looking  metric  to  implement 
reduced  MICAP  hours.  However,  it  is  imperative  to  also  look  at  the  efficiency  of  the 
program  and  not  just  the  effectiveness. 

Model  4  introduces  cost  data  into  Model  3  by  setting  the  actual  replacement  cost 
of  the  NSN  as  the  variable  coefficient.  Using  the  same  business  rules  from  Model  3,  the 
Model  4  output  contained  only  one  variable  from  the  Model  3  output,  XI 520,  and  an 
additional  seven  variables,  in  values  ranging  from  1  to  4  each,  as  shown  in  Table  11. 
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Table  11:  Model  4  Output  (FY12) 


Variable 

NSN 

Amount 
(per  month) 

X1301 

1560012075267 

1 

X1520 

1560016030195 

1 

X2366 

2840010753644 

1 

X2769 

3130008092769 

4 

X3338 

4730011940163 

2 

X5052 

5330002211231 

1 

X6084 

5930007229740 

1 

X6711 

6130014612915 

1 

TOTAL 

12 

By  inserting  cost  into  Model  4,  the  objective  function  of  the  linear  program  now 
produces  more  useable  information,  as  it  introduces  efficiency  into  the  model  to 
complement  the  effectiveness  from  Model  3.  This  provides  an  alternate  intent  of 
minimizing  actual  cost  for  efficiency.  Fortunately,  Model  4  successfully  optimizes  based 
on  the  constraint  of  reducing  MICAP  hours  by  the  prescribed  amount  (25%  reduction), 
making  Model  4  effective.  So,  although  the  output  from  Model  4  increases  the  total 
amount  of  items  to  place  into  stock  from  7  to  12  (as  shown  in  Tables  9  and  10),  it  reached 
the  optimal  solution  for  both  effectiveness  and  efficiency,  based  on  the  given  constraints. 
(Note:  no  correlation  is  being  made  on  the  effect  of  Model  4  on  TNMC-S  or  AA  rates.) 
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Forward-Looking  Metric:  Cost  to  Decrease  Recurring  MICAP  Hours 

To  further  develop  a  forward-looking  metric,  the  Model  4  optimization  linear 
program  was  run  a  series  of  times  to  plot  how  to  decrease  recurring  MICAP  hours.  This 
was  done  through  the  following  steps,  resulting  in  the  plot  at  Figure  10. 

Step  1:  The  median  of  the  FY 12  data  was  found  for  recurring,  non-recurring  and 
in-inventory  MICAP  hours,  as  follows: 


Table  12:  FY12  Median  MICAP  Hours 


Median  MICAP  Hours 

Recurring  MICAP  Hours 

13,491.4 

Non-Recurring  MICAP  Hours 

40,780.5 

In-Inventory  MICAP  Hours 

214.5 

Step  2:  Using  the  median  Recurring  MICAP  hours  (13,491.4  hours)  as  the 

maximum  amount  the  MICAP  hours  can  be  reduced,  Model  4  was  run 
and  the  cost  obtained  from  the  objective  function. 

Step  3:  The  Recurring  MICAP  hour  constraint  was  incrementally  reduced  by 
1000  hours  and  cost  data  obtained,  until  the  output  was  zero. 

Step  4:  The  stable  medians  for  non-recurring  and  in-inventory  MICAP  hours  and 
the  decreasing  recurring  MICAP  hours  were  plotted  with  the  applicable 
cost  for  Recurring  MICAP  hours,  as  shown  in  Figure  10  (cost  shown  in 
millions  of  dollars). 
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Figure  12:  Cost  ($M)  to  Decrease  Recurring  MICAP  Hours 

Summary 

Building  onto  Model  2.1,  Models  3  and  4  were  developed  to  introduce  the  most 
effective  and  efficient  way  to  minimize  cost  will  improving  MICAP  hours.  The  outcome, 
as  shown  in  Figure  10,  shows  the  cost  associated  with  decreasing  Recurring  MICAP 
hours  based  on  Model  4,  using  an  integer  linear  program  and  optimizing  for  actual  cost. 
This  forward-looking  metric  can  be  used  to  effectively  plan  to  decrease  MICAP  hours  in 
the  most  efficient  manner. 
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V.  Conclusions  and  Recommendations 


Chapter  Overview 

This  research  began  to  answer  the  question  “Can  forward-looking  metrics  be  used 
in  Air  Force  sustainment  operations  and  if  so,  how?”  Specifically,  this  research  focused 
on  developing  a  forward-looking  metric  for  Air  Force  sustainment  operations  by 
examining  recurring  MICAP  hours  and  optimizing  stock  levels  via  linear  programming. 
The  following  investigative  questions  were  analyzed: 

1.  What  metrics  are  currently  used  for  sustainment  operations  in  the  Air  Force? 

2.  What  are  forward-looking  metrics? 

3.  Can  forward-looking  metrics  be  developed  to  mitigate  current  sustainment 
operation  shortfalls? 

4.  How  can  forward-looking  metrics  be  used  to  optimize  stock  levels,  within  a  given 
set  of  constraints,  based  upon  a  changing  MICAP-hour  requirement? 

Conclusions  and  Significance  of  Research 

The  intent  of  this  research  was  to  develop  a  forward-looking  metric  that  can  be 
used  to  determine  how  to  change  stock  levels  to  reduce  MICAP  hours  within  the  Air 
Force  supply  system.  A  forward-looking  metric  was  developed  via  an  integer  linear 
program  using  C-5  fleet  data.  Figure  1 1  depicts  the  mathematical  model’s  optimal 
solutions  of  reducing  recurring  MICAP  hours  at  the  associated  costs.  It  identifies  the 
most  effective  means  (recurring  MICAPs)  and  provides  actual  NSNs  and  associated 
quantities  to  buy.  This  research  has  successfully  answered  the  question,  that  Yes, 
forward-looking  metrics  CAN  be  developed  for  Air  Force  sustainment  operations. 
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Although  this  is  only  one  area  of  sustainment  operations,  this  same  methodology  could  be 
applied  across  multiple  areas  and  weapon  systems  within  the  Air  Force’s  control. 

This  research  is  significant  for  three  reasons.  First,  “forward-looking  metric” 
until  this  point  has  merely  been  used  as  a  buzz-word.  Although  the  AFSC  posed  the 
question,  no  extensive  research  has  been  done  and  it  is  not  currently  in  use  in  the  Air 
Force.  The  preponderance  of  the  current  initiatives,  some  of  which  were  highlighted  in 
this  research,  only  look  at  the  current  metrics  framework  and  try  to  enhance  their 
capability.  For  example,  the  Cascading  Metrics  team  has  identified  a  need  for  two  new 
metrics,  but  both  are  historical  metrics  and  do  not  encapsulate  any  forward-looking 
capability.  As  technology  enables  the  Air  Force  to  data  mine  metrics  more  efficiently  and 
effectively,  the  Air  Force’s  supply  chain  managers  must  begin  to  incorporate  new  ways 
of  analyzing  itself;  forward-looking  metrics  provide  that  capability. 

Second,  this  research  was  able  to  provide  an  actual  forward-looking  metric 
program  for  decreasing  recurring  MICAP  hours.  Further,  this  program  can  be  easily  used 
for  any  of  the  Air  Force’s  weapon  systems  and  potentially  extrapolated  into  other 
sustainment  operations.  From  this,  a  decision  maker  can  determine  a  future  optimal  level 
of  capability  for  sustainment  operation.  In  action,  a  decision  maker  can  use  knowledge  of 
future  increased  operations,  determine  the  appropriate  level  of  support  (as  defined  within 
this  research  as  the  allowable  monthly  MICAP  hours  for  an  aircraft  fleet),  and  increase 
specific  stock  levels  (NSNs)  to  meet  that  needed  requirement.  This  research  provided  an 
actionable  metric  and  tool. 

Lastly,  this  research  posed  a  new  way  to  look  at  MICAP  hours  by  sub-diving  the 
hours  among  three  categories:  recurring,  non-recurring  and  in-inventory  MICAP  hours. 
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Typically,  the  Air  Force  looks  as  MICAP  hours  as  one  entity,  which  makes  it  difficult  to 
determine  which  assets  can  enhance  capability  the  most.  For  example,  if  only  the  highest 
MICAP  hours  are  looked  at,  then  the  Air  Force  might  mistakenly  focus  on  a  non¬ 
recurring  MICAP  that  potentially  grounded  the  entire  fleet.  The  MICAP  hours  would  be 
extremely  high,  but  there  is  no  value-added  in  increasing  stock  levels  for  these  assets  as 
there  isn’t  a  recurring  requirement.  This  would  also  be  a  great  area  of  additional 
research,  to  determine  how  to  incorporate  the  different  categories  into  additional  metrics, 

Recommendations  for  Action 

It  is  recommended  that  AFSC  further  develop  this  forward-looking  capability  into 
software  that  can  be  used  across  the  enterprise.  Potentially,  the  optimization  program 
developed  in  this  research  could  be  used  as  ESCAPE  begins  software  development  for  its 
metrics  capability.  Access  into  the  current  Air  Force  data  warehouses  and  programs  need 
to  be  directly  and  electronically  accessed  to  feed  into  the  program  for  ease  of  use  and 
continual  updating.  Additionally,  this  research  be  applied  across  all  the  weapon  systems 
for  use,  as  it  can  function  across  any  platform  that  captures  and  stores  the  same  data. 

Recommendations  for  Future  Research 

Due  to  the  wide  breadth  of  the  initial  problem,  this  research  was  significantly 
narrowed  down  to  look  at  only  one  aspect  of  sustainment  operations,  stock  levels. 

Further,  only  basic  criteria  for  optimizing  stock  levels  based  on  recurring  MICAP  hours 
were  used  to  keep  the  scope  of  the  research  attainable.  Therefore,  there  are  significant 
opportunities  for  follow-on  research,  as  follows: 
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Using  the  current  research  as  a  basis,  include  Demand  (parts  ordered)  for  the 
NSNs  and  how  Stock  Levels  were  changed  over  time.  If  demand  remains 
constant  and  a  NSN  has  a  steady  MICAP  rate  (hence  a  recurring  MICAP),  the 
stock  levels  were  likely  increased  to  meet  that  requirement.  This  will  be  able 
to  provide  a  more  accurate  trend  of  the  requirement,  vice  using  only  the 
MICAP  hours. 

Using  the  current  research  as  a  basis,  incorporate  TNMC-S  and  AA  rates  into 
the  optimization  program.  Since  AA  is  the  overall  indicator  for  sustainment 
operations  success,  this  may  highlight  other  NSNs  that  drive  effectiveness 
higher. 

Find  new  ways  to  incorporate  the  different  categories  of  MICAPs  (recurring, 
non-recurring  and  in-inventory)  into  sustainment  operations  and  metrics. 

Determine  the  correlation  between  AA  and  the  Cascading  Metrics  hierarchy, 
as  presented  by  the  635  SCOW.  This  research  topic  was  presented  by  the 
participants  in  the  December  2012  IPT  and  would  be  a  useful  addition  to  the 
hierarchy  as  it  evolves. 

Using  the  current  research  as  a  basis,  develop  data  to  address  the  minimum 
MICAP  hour  increase  due  to  decreased  funding  to  this  mission  area. 


Summary 

Can  Forward-Looking  Metrics  be  used  within  the  Air  Force’s  sustainment 
operations?  Yes,  they  can,  and  this  research  has  developed  an  initial  methodology  and 
linear  program  for  a  forward-looking  metric.  The  metric  developed  can  determine  the 
optimal  stock  level  increases  to  improve  (decrease)  monthly  MICAP  hours,  subject  to  the 
lowest  cost  with  the  highest  effectiveness.  Using  the  C-5  Galaxy  aircraft  fleet’s 
sustainment  data,  an  integer  linear  program  can  be  manipulated  to  determine  which  parts 
(by  NSN)  should  be  attained  to  decrease  the  MICAP  hours  and  at  which  levels.  This 
research  proves  forward-looking  metrics  are  a  viable  tool  for  the  future  of  sustainment 
operations  and  should  be  leveraged  in  combination  with  technology  to  be  the  most 
effective  and  efficient  Air  Force  in  the  world. 
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Appendix  A:  Cascading  Metrics  (635  SCOW,  2012) 
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Appendix  B:  Table  of  Variables  and  NSNs  used  in  Model  1 


Variable 

NSN 

X001 

1560000817000 

X002 

1560000855872 

X003 

1560001758108 

X004 

1560001957356 

X005 

1560001957357 

X006 

1560002251772 

X007 

1560003175310 

X008 

1560004048394 

X010 

1560004114743 

X011 

1560004114744 

X012 

1560004228335 

X013 

1560004228553 

X014 

1560004502244 

X016 

1560004553200 

X017 

1560004553201 

X020 

1560004814504 

X021 

1560004851725 

X022 

1560004981148 

X025 

1560007634932 

X026 

1560007742114 

X028 

1560010052521 

X029 

1560010052522 

X030 

1560011492746 

X031 

1560011694763 

X032 

1560011694764 

X033 

1560012575349 

X035 

1560012611304 

X036 

1560012611305 

X037 

1560012611306 

X038 

1560012611307 

X039 

1560012611308 

X040 

1560012611309 

X041 

1560012611310 

X042 

1560012611311 

X043 

1560012611312 

X044 

1560012611313 

X045 

1560012611314 

X046 

1560012611315 

X047 

1560013131802 

X048 

1560014372544 
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X049 

1560014372548 

X050 

1560014372557 

X051 

1560014375212 

X058 

1650002388623 

X059 

1650002411858 

X060 

1650002418858 

X062 

1650004169660 

X063 

1650004176300 

X066 

1650004312559 

X067 

1650004312578 

X068 

1650004866297 

X069 

1650004877678 

X070 

1650004884600 

X073 

1650005350668 

X074 

1650008318134 

X075 

1650008335289 

X076 

1650008889805 

X077 

1650008900210 

X079 

1650010751690 

X080 

1650012115433 

X081 

1650012461515 

X082 

1650012481754 

X083 

1650012488592 

X084 

1650013271355 

X085 

1650014434863 

X087 

1660001647347 

X089 

1680001140352 

X090 

1680001851139 

X092 

1680002487639 

X095 

1680003437650 

X097 

1680004024827 

X098 

1680004029812 

X099 

1680004122737 

X101 

1680007580025 

X102 

1680007607881 

X103 

1680011627371 

X104 

1680011640384 

X105 

1680011664022 

X106 

1680011775069 

X107 

1680011798080 

X108 

1680011815672 

X109 

1680011815673 

X110 

1680012133841 

Xlll 

1680012133842 
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X112 

1680012164525 

XI 14 

1680994832212 

X116 

3120011899293 

XI 17 

4210004973144 

X119 

4310012534928 

X120 

4810001452615 

X121 

4810001506089 

X122 

4810002399239 
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Appendix  C:  Table  of  Variables  and  NSNs  used  in  Model  2 


Variable 

NSN 

X014 

1560004502244 

X021 

1560004851725 

X028 

1560010052521 

X029 

1560010052522 

X031 

1560011694763 

X032 

1560011694764 

X035 

1560012611304 

X042 

1560012611311 

X047 

1560013131802 

X048 

1560014372544 

X049 

1560014372548 

X050 

1560014372557 

X068 

1650004866297 

X082 

1650012481754 

X098 

1680004029812 

X119 

4310012534928 

X122 

4810002399239 
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Appendix  D:  Table  of  Variables  and  NSNs  used  in  Model  2.1  (FY10) 


Variable 

NSN 

X0186 

1560001363085 

X0213 

1560001441040 

X0244 

1560001705154 

X0348 

1560002410999 

X0479 

1560004097064 

X0563 

1560004502244 

X0675 

1560004851725 

X0846 

1560007253356 

X0851 

1560007254566 

XI 043 

1560008646976 

X1058 

1560008718185 

X1059 

1560008718186 

XI 126 

1560010052521 

XI 127 

1560010052522 

XI 143 

1560010144483 

X1234 

1560011694763 

X1235 

1560011694764 

XI 240 

1560011744032 

X1351 

1560012611304 

X1358 

1560012611311 

X1397 

1560013131802 

X1429 

1560014372544 

X1432 

1560014372548 

X1433 

1560014372557 

X1441 

1560014536213 

X1483 

1560015543005 

X1484 

1560015543006 

X1485 

1560015543063 

X1486 

1560015543064 

X1613 

1620010805925 

X1631 

1620012056281 

XI 640 

1620014621373 

X1650 

1630011826267 

X1729 

1650002307141 

XI 747 

1650003238292 

X1822 

1650008563281 

X1858 

1650012481754 

X1861 

1650012599382 

X2045 

1680002417901 

X2084 

1680004024827 
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X2085 

1680004029812 

X2171 

1680008333945 

X2283 

1680013465943 

X2333 

2620007022972 

X2343 

2835013429919 

X2406 

2915004899215 

X2416 

2915011896019 

X2418 

2915012147308 

X2811 

4140001060997 

X2859 

4510004101062 

X2880 

4510015732420 

X3121 

4730000950949 

X3404 

4810000099505 

X3408 

4810001162144 

X3413 

4810001360476 

X3431 

4810004103629 

X3443 

4810004866293 

X3469 

4810012603664 

X3549 

4820011916897 

X3562 

4820013181730 

X3580 

4920014688951 

X4502 

5315001922153 

X5131 

5330005861070 

X5499 

5340002021511 

X5889 

5821014563702 

X5906 

5826014120738 

X5938 

5841015204271 

X5962 

5895011925440 

X5974 

5895014904756 

X6681 

6105012293758 

X6689 

6110011614873 

X6710 

6130014223698 

X6714 

6130015357275 

X6860 

6220007088861 

X6994 

6610010261701 

X7136 

6685005267864 

X7144 

6685008037705 

X7149 

6685008908410 
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Appendix  E:  Table  of  Variables  and  NSNs  used  in  Model  2.1  (FY11) 


Variable 

NSN 

X0186 

1560001363085 

X0213 

1560001441040 

X0869 

1560007322771 

X1211 

1560011160447 

X1234 

1560011694763 

X1235 

1560011694764 

X1429 

1560014372544 

X1432 

1560014372548 

X1582 

1620003175535 

X1613 

1620010805925 

X1650 

1630011826267 

X1710 

1650001336104 

X1729 

1650002307141 

X1758 

1650004169660 

X1833 

1650010751690 

X1879 

1650014434863 

X1932 

1660004568546 

X2001 

1680001033526 

X2085 

1680004029812 

X2230 

1680011775069 

X2333 

2620007022972 

X2367 

2840010753645 

X2406 

2915004899215 

X2416 

2915011896019 

X2503 

3040008190108 

X2808 

4110011560198 

X3407 

4810001025411 

X3420 

4810001791284 

X3438 

4810004538951 

X3464 

4810012110166 

X3581 

4920015064328 

X3920 

5306001877313 

X5146 

5330007819468 

X5294 

5330015771850 

X5636 

5340015609257 

X5889 

5821014563702 

X5911 

5826014651630 

X5938 

5841015204271 

X5978 

5895015072148 

X6134 

5930015394078 
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X6569 

5985010890737 

X6682 

6105012755704 

X6714 

6130015357275 

X6951 

6350002996203 

X6958 

6605010182181 

X7009 

6610015004464 

X7014 

6610995938834 

X7016 

6610998171137 

X7045 

6615014934075 

X7140 

6685006898672 

X7175 

7010015063625 

X7176 

7010015407428 
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Appendix  F:  Table  of  Variables  and  NSNs  used  in  Models  2.1  (FY12),  3  and  4 


Variable 

NSN 

X0362 

1560002489209 

X0563 

1560004502244 

X0894 

1560007634932 

X0906 

1560007742114 

X1234 

1560011694763 

X1250 

1560011795638 

X1301 

1560012075267 

X1429 

1560014372544 

X1432 

1560014372548 

X1520 

1560016030195 

X1729 

1650002307141 

X1799 

1650004954535 

X1858 

1650012481754 

X1879 

1650014434863 

X1932 

1660004568546 

X1983 

1680000210334 

X2045 

1680002417901 

X2161 

1680007580025 

X2230 

1680011775069 

X2343 

2835013429919 

X2366 

2840010753644 

X2416 

2915011896019 

X2438 

2995001153522 

X2515 

3040013021166 

X2769 

3130008092769 

X2841 

4310012534928 

X3338 

4730011940163 

X3484 

4820001322002 

X5052 

5330002211231 

X5889 

5821014563702 

X5938 

5841015204271 

X5978 

5895015072148 

X6084 

5930007229740 

X6711 

6130014612915 

X6966 

6605015705261 

X7007 

6610014932400 

X7014 

6610995938834 

X7082 

6680004367623 

X7089 

6680011016433 

X7175 

7010015063625 
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